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20 February 1975 


MEMORANDUM FOR THE RECORD: 


SUBJECT: Remarks of Mr. Andrew Marshall 


1. Mr. Marshall met with the Senior Seminar fron 9830 
throuch 1000, Thursday, 15 February to discuss the tonic 
“Intelligence Products: Problems and Improvenents.’' When we 
considered inviting Mr. *iarshall originally, it was with the 
idea of hearing nim talk on his work in the field of net 
assessments aS a subject fittines the week's theme of "New 
Methodolosies of Interest to Intelligence.’ When Mr.[._ |] STAT 
met with him, however, ‘fr. Marshall. suggested that he concen- 
trate instead on the products of intellisence to which we agree 
in view of recent changes. Accordingly, while Mr. Marshall 
eccasionally referred to new methods, it was in the context of 
exhortins CIA to exneriment with them, rather than the substance 
of net assessment as a new technique. 


2. Mr. Marshall began by saying that in his talk he would 
encourage us to employ new techniques to a sreater extent, but 
that he would also address the question of the difficulties which 
arise in trying to utilize them. He then Launched into a 
lengthy <iscussion of the intellisence product as he has per- 
ceived it from the vantase point of nis earlier associations with 
the Agency and the Intelligence Community while at RAND and parti- 
cularly From the perspective of nis present position as chief of 
the NSC Intelligence Committee's Net Assessnent Groun. fis 
remarks about the product were critical in varying degrees and 
are based, accordins to Mr. Marshall, on comments he has reccived 
from “consumers whon he specifically identified as "members" of 
the NSC staff under Dr. Kissinger. 


3. His criticisms are as follows: 


a. The intelligence product and the production 
organizations are deficient in their “communication.” In this 
connection he mentiened writings style. He said we do not experi- 
ment enough with different formats. He singled out several tines 
the lack of adequate summaries to accompany longer reports and 
critized our summaries as “not hard hitting enough," i.e., they 
do not summarize the conclusions of the longer renort well or 
Firmly enough. ie mentioned that “uncertainties” concerning the 
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judgments and estimates in the intelligence product are not 
adequately specified for the benefit of the reader. He cited 
the "anonymity" of authorship as a major problem on the grounds 
that a consumer comes to trust an intellisence statement on the 
basis of knowing the individual whose judement is expressed. 


b. The second set of criticisms concerned the ‘focus 
of the product.’ This ‘ir. Marshall characterized as writing 
about the right suhjects in the risht way. He stated that he 
recognizes that intelligence producers do not get much feedback 
--and that which is given is skewed in favor of praise. He 
assigns blame to the consumer who is busy and does not know 
what he wants, but particularly to the producer. He criticized 
the Agency for failing to do "market research." He agreed that 
the questionnaires techniques is probably no good, but referred 
several tines to the ‘Madison Avenue" approach. He said that we 
Should be more forceful and make a greater effort to find out 
how we can help the consumer--again pointing to personnel contact 
between the area men in the NSC staff and area exnerts in the 
Agency. With regard to focus, he also said he believes we should 
try more experimental products, the same products in a variety of 
formats and particularly try to experiment with new methodolosgies. 


c. ir. Marshall then cited some "Oreanization and 
Management Changes" which he opined are necessary in order to cet 
new approaches off the ground. He said that it has proven most 
difficult to set exrerimental efforts gcing because on-line pro- 
duction offices are reluctant to put sood people on problens 
being dealt with in an experimental mode. ile suggested that a 
separate unit(s) be set up for the sole purpose of experimentation 
in the field of intellisence productioa. He cited trainings of 
analysts to work in the field of new methodologies as important. 


d. He then turned to the "quality of analysis.” ile 

- Stated that much of cur analysis is wanting in interpretation, 
not just occasional holes, but with respect to "whole programs." 
He charged that we fail te answer vital questions of "Why?" or 
"What Next?" and that we fail to put forth adequate hypotheses. 
He noted that there is not much in the way of reasoning behind 
our intelligconce statements and that they cannot be compared 

at a future date. In sum, he claimed, we have provided few 
"basic answers" over the 190 to 20 years of our existence. In 

a provocative assertion, he clained that in order to provide 
answers to fundanefital questions, he could do almost as well bv 
drasging in a bunch of "people off the street." We just are not 
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good enough. As a corrective, he urged we adopt the use of 
footnotes citing sources and append annexes which lay out the 
reasoning behind intelligence statoments. He repeated that it 
is difficult to find "real analysis" in our products. He also 
Suggested that we try harder to reach "our real readers" who, 
he suggested, are the staff persons working for top leaders, 
€.g., the NSC staff. He again stressed the importance of good 
summaries for top readers. He suggested that we experiment 
more with visual, rather than word, presentations of material, 
e.g., cartoons. To improve the analytical quality of our pro- 
duct, he proposed that we institute an R&D effort in analysis, 
giving in-depth study to selected topics and that we make an 
effort to develop means of conducting a critical review on a 
Sampling basis in order to get feedback. 


e. Turning to new methods, he held up Bayesian 
analysis as providing a systematic way of handlins uncertainties, 
citing Dr. Kissinger's comment to him that policy-making is 
"betting on the future.’"’ Mr. Marshall sugsested, that we might 
try expressing our intelligence judgments to Dr. Kissinrer in 
the form of betting odds. He also mentioned the Bureaucratic 


otentially useful in gaining 


insicht into the ti r G aders, noting the 
- STAT as helpful. 


3. <As to "cautionary tales," Mr. Marshall gave his vicw 
that game theory has not been very useful. He cited the barriers 
that exist to innovation, giving as an example a rather lengthy 
effort on his part going back to 1958 while he was at RAND to 
get the Agency to take a new tack in analyzing Soviet weapons pro- 
grams. 


he 
Ss 


4. Mr. Marshall stated, in summary, that he feels that 
Agency should take the leadership in certain key areas vis-a-v 
universities and think-tanks. Ile said in some cases these are 
areas in which CIA is falling behind. He urged that the Asency 
have closer contact with this outside work. He observed that we 
should not expect this work to be developed outside to the extent 
that it then be handed to CIA and immediately applied. 
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5. The question and answer session served mainly to 
amplify some of ‘Mr. Marshall's statements. One exchange con- 
cerned the unwillingness of the consumer to "Jevel" with CIA 
--saying only good things to us, put voicing dissatisfaction 
to others such as Marshall. He again implied that the burden 
of producer-consumer communication falls on the intelligence 


producer. 
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